Optimally-Weighted Herding is Bayesian Quadrature
نویسندگان
چکیده
Herding and kernel herding are deterministic methods of choosing samples which summarise a probability distribution. A related task is choosing samples for estimating integrals using Bayesian quadrature. We show that the criterion minimised when selecting samples in kernel herding is equivalent to the posterior variance in Bayesian quadrature. We then show that sequential Bayesian quadrature can be viewed as a weighted version of kernel herding which achieves performance superior to any other weighted herding method. We demonstrate empirically a rate of convergence faster than O(1/N). Our results also imply an upper bound on the empirical error of the Bayesian quadrature estimate.
منابع مشابه
Weighted quadrature rules with binomial nodes
In this paper, a new class of a weighted quadrature rule is represented as -------------------------------------------- where is a weight function, are interpolation nodes, are the corresponding weight coefficients and denotes the error term. The general form of interpolation nodes are considered as that and we obtain the explicit expressions of the coefficients using the q-...
متن کاملActive Learning of Model Evidence Using Bayesian Quadrature
Numerical integration is a key component of many problems in scientific computing, statistical modelling, and machine learning. Bayesian Quadrature is a modelbased method for numerical integration which, relative to standard Monte Carlo methods, offers increased sample efficiency and a more robust estimate of the uncertainty in the estimated integral. We propose a novel Bayesian Quadrature appr...
متن کاملSuper-Samples from Kernel Herding
We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting “kernel herding” algorithm is an infinite memory deterministic process that learns to approximate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rateO(1/T )which is much faster than the usual O(1/ √ T ) for iid ...
متن کاملOptimally Blended Spectral-Finite Element Scheme for Wave Propagation and NonStandard Reduced Integration
We study the dispersion and dissipation of the numerical scheme obtained by taking a weighted averaging of the consistent (finite element) mass matrix and lumped (spectral element) mass matrix for the small wavenumber limit. We find and prove that for the optimum blending the resulting scheme (a) provides 2p+4 order accuracy for p-th order method (two orders more accurate compared with finite a...
متن کاملSampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in probabilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute the marginal likelihood or a ...
متن کامل